Automating GCP Setup with Terraform

The nOps GCP API Enablement Terraform Module automates the process of enabling specific Google Cloud Platform APIs across all projects in an organization. This ensures consistency and saves significant time compared to manual enablement.

Repository: github.com/nops-io/terraform-gcp-nops-integration

Overview

This module enables the APIs required for full nOps visibility and grants necessary IAM roles.

API Service	API Service ID	Scope
Cloud Asset API	cloudasset.googleapis.com	Central Ingestion Project
Cloud Billing API	cloudbilling.googleapis.com	Central Ingestion Project
Recommender API	recommender.googleapis.com	Central Ingestion Project
BigQuery Reservation API	bigqueryreservation.googleapis.com	Optional (only for flat-rate BigQuery pricing)

IAM Roles Granted

Role	Scope	Purpose
Service Usage Consumer	Central Ingestion Project	Allows nOps to route API calls through your project
BigQuery Resource Viewer	Central Ingestion Project	(Optional) Only if using flat-rate BigQuery pricing

Simplified Setup

The Cloud Asset API can discover Cloud SQL, Cloud Run, GKE, and other resources across your entire organization from the central project. This means you don't need to enable individual service APIs in every project.

Prerequisites

Before using this module, ensure you have the following:

• Terraform (>= 1.0) or OpenTofu installed.
• Billing Enabled: Billing must be enabled for the central ingestion project.
• Google Cloud Credentials with the following permissions:
  • resourcemanager.projects.list - to list all projects in the organization.
  • serviceusage.services.enable - to enable APIs.
  • serviceusage.services.get - to check API status.
  • Organization-level or project-level admin role.

Installation

Using Terraform / OpenTofu

1. Clone or download the module:

   git clone https://github.com/nops-io/terraform-gcp-nops-integration.git
   cd terraform-gcp-nops-integration

Usage

Basic Example

Create a main.tf file in your working directory:

terraform {
  required_version = ">= 1.0"
  
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.0"
    }
  }
}

provider "google" {
  # Option 1: Use Application Default Credentials (recommended)
  # Run: gcloud auth application-default login
}

module "enable_gcp_apis" {
  source = "github.com/nops-io/terraform-gcp-nops-integration"
  
  organization_id              = "123456789012"  # Your GCP Organization ID
  central_ingestion_project_id = "my-central-project-id"
  
  # All APIs are enabled by default.
  # To customize, you can override defaults:
  # enable_cloud_asset_api = false
}

Running the Module

1. Initialize Terraform:

   terraform init
2. Review the Plan:

   terraform plan
3. Apply Configuration:

   terraform apply

Authentication

Option 1: Application Default Credentials (Recommended)

gcloud auth application-default login

This uses your user credentials. Ensure you have the necessary permissions.

Option 2: Service Account Key

1. Create a service account with required permissions.
2. Download the JSON key file.
3. Set the path in your provider configuration:

   provider "google" {
     credentials = file("path/to/service-account-key.json")
   }

Finding IDs

Finding Your Organization ID

gcloud organizations list

Finding Your Central Ingestion Project ID

The central_ingestion_project_id is the project where Cloud Asset and Billing APIs will be enabled. It should be an existing project you have admin access to.

gcloud projects list

Use the PROJECT_ID column.